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Abstract 

We have recently constructed a piecewise quadratic Lyapunov function to prove the boundedness of 
the reachable values set of piecewise affine discrete-time systems. The method developed also provided 
an overapproximation of the reachable values set. In this paper, we refine the latter overapproximation 
extending previous works combining policy iterations with quadratic Lyapunov functions. 


1 Introduction 


Several catastrophic events showed the importance of the formal verification of programs. Some of these 
failures are caused by overflows. A method to prove the absence of overflows in numerical programs consists 
in providing precise safe bounds over the reachable states of the program variables. 

In this paper, we are interesting in a particular class of numerical programs: single while loop programs 
with a switch-case structure inside the loop body. Moreover, we suppose that test and assignment functions 
are affine. These programs can be represented as piecewise affine discrete-time systems. To overapproximate 
the reachable states of the program variables is thus reduced to overapproximate the reachable values set of 
a piecewise affine discrete-time system. Hence, we propose to compute automatically precise bounds over 
piecewise affine discrete-time systems using policy iterations and piecewise quadratic Lyapunov functions. 

Initially policy iteration solves stochastic control problems |How60| which can be reduced to solve fixed 
point problems involving functions with maxima of affine functions coordinates. Policy iteration was then 
extended to zero-sum two-player stochastic games |HK66| . this extension allows the computation of the 
unique fixed point of min-max of affine maps. The very first extension of policy iterations in program anal¬ 
ysis was in 2005 by Costan et al |CGG~*~05l . Since then, the usage of policy iteration in various verification 
problems greatly increases: in |GSA~*~12 , the authors describe policy iteration algorithm to overapproximate 
the reachable values set of numerical programs with affine assignments; in |Masl2| , the author proves termi¬ 
nation by policy iteration; in |SS13I ISJVGlf] the authors propose to embed policy iterations for programs 
dealing with both numerical and boolean variables. 

The method developed in |AG15| allows to prove that the reachable values set of a piecewise affine system 
is bounded. The method relies on the synthesis of a piecewise quadratic Lyapunov function of this piecewise 
affine system. The problem formulation makes appear as a decision variable an upper bound on the Euclidian 
norm of the state variable. This upper bound can be very loose since it combines all the coordinates together. 
We propose to use a templates based method. A templates method consists in representing sets as sublevel 
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sets of given functions called templates. Then to compute an overapproximation is reduced to computing 
bounds over the templates. The most precise overapproximation with respect to these templates is provided 
by the vector of bounds satisfying a smallest fixed point. In our context, the generated piecewise quadratic 
Lyapunov function is used as a template. We complete the templates basis by the square of variables. Finally 
we use policy iterations to solve the (smallest) fixed point equation. Thus, policy iterations algorithm leads 
to tighter bounds over the reachable values set. 

The use of quadratic Lyapunov function as quadratic templates was explicitly done in |R,,TGF12] but it is 
not enough to prove the boundedness of reachable values set of a piecewise affine system unless that a common 
quadratic Lyapunov function exists. Policy iteration algorithms in templates domain proposed in |AGG12I 
lGSA+1^ used quadratic templates and did not handle piecewise quadratic templates. In this paper, we 
adapt policy iteration based on Lagrange duality Adjl4 to piecewise quadratic functions. The works on 


piecewise quadratic Lyapunov functions |Joh03l IMFTMdO] are also related to this paper. Their authors 
are interested in proving stability of piecewise linear systems. However, as classical quadratic Lyapunov 
functions, piecewise quadratic Lyapunov functions provide sublevel invariant sets to the system. We use this 
latter interpretation for a verification purpose. Finally, note that tropical polyhedra domain |A1109| generates 
disjunctions of zones as invariants. The latter invariants did not encode quadratic relations between variables. 

The first contribution of the paper is the formalisation of piecewise quadratic Lyapunov functions to prove 
the boundedness of the trajectories of a piecewise affine discrete-time dynamical system. This formalisation 
uses the theory of cone-copositive matrices which is also an original contribution in this context. 

The main contribution of the article is the extension of policy iterations algorithm to the piecewise 
quadratic Lyapunov functions in order to provide precise bounds on the reachable values. Indeed, policy 
iteration has just been constructed in the case of quadratic functions. 


Notations 

Numbers. N denotes the set of nonnegative integers, then for d S N, [d] = {1,..., d}. K is the set of reals, 
M+ the set of nonnegative reals and denotes the set of vectors of d reals. We denote by the set of 

subsets of 

Inequalities. For y,z y < z (resp. y < z) means VZ S [d], ?/; < z/, (resp. VZ S [d], yi < zi) and 

y ^w,s z is a, mix of weak and strict inequalities. 

Matrices. M„xm is the set of matrices with n rows and m columns. On,m and 0„ are respectively the null 
matrices of M„x 7 n and M„xn- Id„ is the identity matrix of M„xn- AT'' is the transpose of M € M„xm- Sn 
is the set of symmetric matrices of size n x n. A ^ 0 means that A is semi-definite positive i.e. A GSd and 
Vcc S x'^Ax >0. is the convex cone of semidefinite positive matrices. 


2 Piecewise affine discrete-time systems 

In this section, we detail the systems we will consider in the paper. 

Piecewise affine systems (PWA for short) are defined as systems the dynamic of which is piecewise affine 
and thus the dynamic is characterized by a polyhedral partition and a family of affine maps relative to this 
partition. For us, a polyhedral partition is a family of convex polyhedra such that: 

y X* = and yi,j Gl, i^jX^nX^=(d . (1) 

iex 

The convex polyhedron X® can contain both strict and weak inequalities and is represented by T* G M„.xm 
and c® G ffi"®. We denote by T® (resp. T^) and c® (resp. the parts of T® and c® corresponding to strict 
(resp. weak) inequalities: 

X® = y e M®' 

= I a; e 


T’'X <.UJ.S 
Tlx < c\. 


T < \ 


( 2 ) 


2 
















Definition 1 (Piecewise Affine System) A PWA is characterized by the triple {X^,X,A) where: 

• is the polytope of the initial conditions of the form (|^; 

• X := G X} is a polyhedral partition i.e. satisfying 0; 

• A'.= {x^ r{x) = A^'x + 6*, i G 1} where Ad G M^xd o^nd P G 
And satisfies the following relation for all k G N; 

xoGX°, ifxkGXf Xk+i = fixk) ■ (3) 

Let P — {X^,X,A) be a PWA. We now define some tools that we need during the analysis. First we define 
the reachable values set TZ of P: 


TZ = \^ A^(A°), where K{x) = P{x) if x G AT® (4) 

fcGN 

We define the set of possible switches: 


Sw :={(*,j) Gl^ \ ndXP ^0} 
where X^ = A® n /®“\a^ ) . 


Finally, we define the set of indices of polyhedra of X which meet the polyhedron of possible initial conditions: 

In := {i G I I A®° p 0} where A®° = A® n A° . (6) 

We introduce for i gX, the following matrix of M((i_|_i)x(d+i): 


F® = 



OlX(i\ 

A® j 


(7) 


Eq. ^ can be rewritten as (l,Xfe+i)‘'' = F®(l,Xfe). 

We are interested in computing automatically precise overapproximation of TZ. We propose to compute 
an overapproximation of 7?. as a set S C'MA such that A° C S and ViGX, xGS’HA® A®x + 6® G S'. 

The set S can be computed as a sublevel of a Lyapunov function containing the initial states. 

From now, we work with a fixed PWA P = (A°, X, A), where A°, X and A are of the form of Def. 


3 Piecewise quadratic Lyapunov functions 


In this paper, we use piececewise quadratic Lyapunov functions for piecewise affine systems to compute 
directly an overapproximation of reachable values set. 

Let 5 be a quadratic form i.e. a function such that for all y G q{y) = y'^ Aqy + h^^y + Cq where Aq G Sd, 
bq G K®* and Cq G K. We define the lift-matrix of g, the matrix of S^+i defined as follows: 


M(Aq,bg,Cq) = M(g) 


{ Cq {bq/2y\ 

\ibq/2) A, J 


( 8 ) 


It is obvious that the 
all X G 


q 1 -^ M(( 7 ) is linear. Let A G M^xd, b G K®^, 
g(Ax + 6)=Q 


and g be a quadratic form, we have, for 



( 9 ) 


Lemma 1 Let A G Sd, 6 G K®* and c G K. Then: (ff y G K®^, y'^ Ay + b'^y -I- c > 0) M(A, b, c) G 
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Definition 2 ((Cone)-copositive matrices) Let M G M^xd- ^ matrix Q GSd which satisfies 

My > 0 y'^Qy > 0 

is called M-copositive. 

An lAd-copositive matrix is called a copositive matrix. We denote hy (M) the set of M-copositive 
matrices and Cd the set of copositive matrices. 

For P G M„xm and c G M", we define the following matrix: 

H(^=(c “i'p ) e M(„+I)x(™+1) (10) 


Lemma 2 Let P e 


and c G M". Then, for all x G M", Px < c 


H(p,c)(;i >0. 


Lemma 3 Let q : —)■ be a quadratic function. Let M G M^xd P G us consider 

C = {x \ Mx < p}. Then M(g) G Cd+i {M,p)^ {q{x) >0, Wx G C). 

and we introduce the following matrices: 


Vi ex, = U{T\P) , 
\fii,j)Gl\ =il 

Vz e In, = 



(lla) 

(llb) 

(llc) 


Lemma 4 For all i G T, C {x \ E’‘(l x'^' fi > 0}, for all {i,j) G Sw, X'‘^ C {x \ E'‘^{1 a;''')''’ > 0} and for 

all i G In, C {x \ E^°{1 a;T)T > Q}. 

Definition 3 (PQL functions) A function L is a piecewise quadratic Lyapunov function (PQL for short) 
for P if and only if there exist a family {(P*, g*), P* € S^, g* e i Gl} and two reals a and fi such that: 

1. Vz e I, Vx e XV L{x) = Lfix) = x'^P^x + 2a;TgV- 

2. ViGl: 

M(PV2gV-a)-M(Id,0,-/3)eCd+i(P*) ; (12) 

5. V (z,j) e Sw.- 

M(PV2gV0)-P*'^M(pl,2g^0)P*eCd+i(P*^) ; (13) 

4 . Vi G In.- 

-M{P\ 2 q\-a)GCd+i{E^°) . (14) 

Theorem 1 (Bounded trajectories) Assume that P admits a PQL function characterized by {(P*, q^),P^ G 
Sd, 9* e z e X} and reals a and fi. Let i G I, Sf = {x G X'^ \ L'^{x) < a} = {x e X* | x"^P^x-\-2x'^q^ < a} 
and S = UigxPV- Then, P C S' C {x e | ||x||2 < fi}. 
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Proof 1 First, we prove that S' C {a; g | ||a :||2 < P}. Let z € X and x G X®. From Eq. (12), Lemma 
and Lemma^ x'^P^x + 2x'^ — a — ||a ;||2 + /3 > 0. This is equivalent to j3 — ||a ;||2 > a — x'^P^x — 2x'^q^ which 
implies that S C {x € | ||a :||2 < fd}. 

Now, we have to prove P Q S. From Eq. we have to prove that for all k gN, A'^(X^) C S. We prove 
it by induction on k. Let x G Since X satisfies (Q, there exists a unique i G In such that xq G X^^. From 
Eq. (14), Lemma^and Lemmaj^ L^(x) < a. Now suppose A^(X°) C S for some k gN. Let y G 
Then y = K{x) for some x G Since X satisfies (Q, there exists an unique (i,j\G Sw such that 

X G (hence y G X^). As x G AT® and x G S, then x G S^. From Eq. (13), Lemma^and Lemma ^ 
0 < L®(a:) — L^(y) = L®(x) — a — {L^{y) — a). As x G Sf, 0 > L®(x) — a which implies that 0 > L^{y) — a 
and finally y G Si, C S . 


3.1 Computational issues 

To construct PQL functions, we are faced with two issues. First, we must know the sets of indices Sw and 
In. Second we have to manipulate cone-copositive constraints. 


3.1.1 The computation of sets Sw and In 

To set Sw is defined from P, the set which we want approximate. To overcome this issue, we consider a 
bigger set by removing the intersection with P: 


Sw:={(z,j)eJ" |lf®^V0} . 


(15) 


Since Ai® and X^ can contain strict inequalities, we can use alternative theorems such as Motzkin’s theo¬ 
rem |Mot51| to compute Sw. Note that we use this technique based LP to determine exactly In. 

The direct application of Motzkin’s transposition theorem |Mot51| yields to the next proposition. 

Proposition 1 Let nG (resp. be the number of strict (resp. weak) inequalities in A'®nA''^. The couple 
{i,j) G Sw if and only if: 




,cl-TiP 


Olxd \ ^ 

-Tl P^A 
-TiWj 



nf + l 

^ pI ^ p" > 0, p > 0 

. k=l 



p = 0 


has no solution. 

Let 7z®q (resp. u^q) be 
only if: 


the number of strict (resp. weak) inequalities in X® H X^. 



"in-1-1 


_ rpi 

^ W 
_2^0 


p = 0 


^ pI = ^, p" >0, p>o 


k=l 


The index i G In if and 


has no solution. 
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3.1.2 Cone-copositive constraints 

Cone-copositive matrix characterizations is an intensive research field and a list of interesting papers about 
can be found in |BSU12| . 

Proposition 2 (Th. 2.1 of [MJ81] i Let M G M^xd- Then: 

{MTCM-tS'lC'eCd and5'eS+}CCd(M) (A) 

If the rank of M is equal to m, then 0 is actually an equality. 

The next proposition discusses simple a characterization of copositive matrices as a sum of a semi-definite 
positive matrix and a nonnegative matrix. 

Proposition 3 ( [Dia6211MM62] ) We have: Vd S N.' -I- C C^;. If d < A then -I- Sj. 


Corollary 1 Let M G M^xd- Then: 


Cd (M) A 


Q G Sd 


3WpG W+ G S+, s.t. 1 
Q - MT {Wp + W+) MhO } 


(*) 


If M has full row rank and d < A, then Q is actually an equality. 

Copositive constraints study is a quite recent field of research. Algorithms exist (e.g. |BD09]) but for the 
knowledge of the author no tools are available. In this paper, in practice, we use Corollary and we replace 
Cd (M) by the right-hand side of Eq. Q. 


3.1.3 Computation of Piecewise quadratic Lyapunov functions using SDP solvers 

Finally, we construct PQL functions using semidefinite programming. We define the notion of computable 
PQL functions. 

Definition 4 (Computable PQL functions) A function L is a computable PQL for to P if and only if 
there exist two reals a and fl and four families: 


• 

V := 

{{P\q^),P^ GSd,q^ 

G iG 1} 

• 

W : = 


X G T}, 

• 

U := 

{(c/;^t7^) GS^O X 

e Sw} 

• 

Z := 

{(z;0,z;0) GS^o X 


such 

that: 




1. 'iiGX, Mx G X\ L{x) 

2. yiGT: 

3. V (i, j) G Sw.- 

4 . yi G In.- 


L^{x) = x'^P'‘x A- 2x'^q’‘; 

M(P% 2qf -a) - M(Id, 0, -13) 

- {Wf + Wf) 

M(P*, 2g*, 0) - 0)F* 

- E^^ (u^ + E^ 

- M{P\ 2q\ -a) - E^^'^ {Zf E Z°f) E 


AO ; 

(16) 

AO ; 

(17) 

A 0; 

(18) 


6 













Let us consider the problem: 


inf 

v,w,u,z, 

06,^ 

s. t. 


Of + /3 


r {V,W,U,Z,a, 

\ a > 0, /3 > 0 


/3) satisfies @ and @ 


(PSD) 


Problem (PSD I is thus a semi-definite program. The use of the sum a -\- j3 as objective function enforces 
the functions L*s to provide a minimal bound /3 and a minimal ellispoid containing the initial conditions. 
The constraint /3 > 0 is obvious since j5 represents a norm. However, a > 0 is less natural but ensures that 
the objective function is bounded from below. The presence of the constraint a > 0 does not affect the 
feasibility. Note that to reduce the size of the problem, we can take = 0 and get an homogeneous PQL 
function. 

Now, we can explain the motivation of (1, Oixd) in Eq. 


( [10| . It would be more natural to express H (P, c) 
as (c — P)- However, when we replace the cone-copositivity constraints by right-hand-side of Eq. 0 and 
by doing this we allow symmetry as it is shown in Example]^ and the vector (1, Oixd) aims to break it. 


Example 1 (Why is there (l,0ixd) in H(P,c)?) Consider X = {x S M | a; < 1}. Let u{x) = (1, x), and 
M = (1 — 1) (H (1,1) without (1, 0) ). Then X = {x | Mu{xy > 0}. 

Now letW >Q and define X' = {x | u{x)M'^WMu{xy > 0}. Since u{x)M'^WMu{xy = Wu{x)M'^Mu{xY = 
2W[l - xf, X' = R fo r all W >0. _ 

Now let us take E = H (1,1) and letW = () with wi, W2, > 0 and define X = {x | u{x)E'^WEu{x)'^ > 

0}. Hence, u{x)E'^ ((JIa (JIa) Eu{xY = Wi + 2^3(1 — x) -I- 1x2(1 — a:)^. Taking for example W2 = Wi = 0 and 
W3 > 0 implies that X = X. 


Proposition 4 Assume that Problem (PSD) has a feasible solution {V,W,U,Z,a,j5). Then: 

1. The family V defines a PQL; 


2. There exists {V,yV,U, Z,a, /3) satisfiying (16), (17) and (18) if and only if Problem (PSD) is feasible; 

3. For all {i,j) € Sw, 


F*'^M(Id,0,0)F* 

^ M(P%2gS-Q;)-kM(0,0,/?) 

On, Oni.n, 




0n„n. + 




4. We have sup ||x ||2 < P; 

x&XO 

5. Lf lfP,W,U,Z,a,j5) is optimal and a > 0 then sup L{x) = a. 

xexo 


Proof 2 Ln appendix. 


4 Sublevel Modelisation 

In Def. 1^ /3 is an upper bound on the Euclidian norm of the state variable. We do not have a precise 
upper bound on each coordinate considered separetely neither a precise upper bound on the state variable 
considering a specific cell. To obtain tigher bounds on the state variables, we intersect Sa with other sublevel 
sets. In |RJGF12l . the authors propose to combine classical quadratic Lyapunov function sublevels and the 
square of variables. In this paper, we apply this technique replacing classical Lyapunov functions by PQL 
functions. Thus we are interested in a set V of the form V = S'q, n Ui^x{y G X* | yf < fi\,l = 1,..., d}. 
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The computation of V is thus reduced to compute /3;. In verification of programs, the method is called a 
templates domain abstraction (for more background |AGG12] 1. 

From Eq. 0- TZ = A{Tl) U We introduce the map F : p(M'^) i—>■ defined by: 

C^F{C) :=A{C)UX° . 

Hence TZ is the smallest fixed point of F in the sense of if C = F{C) then TZ C C. From Tarski’s theo¬ 
rem |Tar55| . since F is monotone on p(IR‘^), then: 

TZ = inf{C e p(M‘’*) I F{C) C C}; (19) 

Consequently, if we take any subset C such that F{C) C C then TZ Q C. We propose to consider a restricted 
family of subsets C parameterized by a; G 

C{uj) := {a; e I Vfc G [d], x\< uJk,L{x) < ujd+i} 


where L is a PQL function of P. We define: 


Vfc G [d], = sup yl and = sup L{y) 

vex° yex« 


We also define for all {i,j) G Sw and for all u) G 


i \2 


VfcG[d], iG"^.^(a;)= sup {Alx + bl) 


Vfce[(i], xl<LLik, 


and 


sup L^iA^x + F) 

V k^[d], x‘^<uJk, 

L'^ {x)<LiJd+i, xGX'^^ 


and finally, we define for all oj G 

VZ G [d -k 1], F/(a;) = sup{ sup_Ft“^. ^(w), Xf} 

(ij)GSw 


and F«(a;) = (F«(cc),...,Fj+i(cc)). 

Proposition 5 The following statements hold: 

1. F{C{uj)) C C{uj) ^ F«(w) < oj; 

2. TZC ^{(^(w) I w G s.t. F*iuj) < to}; 


3. For all I G [d 1]; ^{uj) is the optimal value of quadratic program; 

4- For all k G [d], X^ = max{( inf Xk)‘^,( sup Xk)^} and if L is constructed from an optimal solution 

2 ^ 6 ^“ xexo 

{'P, yV,U, Z, a, (3) of ( PSD[ ) such that a > 0, then = a. 

Proof 3 In appendix. 


5 Policy Iteration Algorithm 

Now, we assume that Problem ( |PSD[ ) has an optimal solution (P, W,U, Z, a, /3) with a > 0 and let L be the 
associated PQL function. 

From Prop.|^ to evaluate F^^ ^{oj) is equivalent to solve a quadratic maximisation problem which is known 
to be NP-Hard ^av90| . So we propose to compute instead a safe overapproximation using Lagrange duality 
and semi-definite programming. 














Proposition 7 Let (i,j) € Sw, I € [(i+ 1], A G The following statements are true: 

Pij,i 

2 - F^ii, and are monotone; 

3. and F^ are upper semi-eontinuous. 

Proof 5 In appendix. 
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To be able to perform a new step in policy iteration, we need a selection property. In our case, the selection 
property relies on the existence of an optimal dual solution. 


Definition 5 (Selection property) Let {i,j) G Sw and I G [d+ 1]. We say that oj G satisfies the 

selection property if there exists A G such that: 

F^/u;) = (23) 

We define: 

SoU ((*,j),/,w) := {A G Kf 1 I Fliiu:) = T;^-,(a;)} 

and 

S := _ 

{uj G I V (z, j) G Sw,VZ G [d+ 1 ],So1a {{i,j),l,uj) 0} . 


Corollary 2 Let {i,j) G Sw, I G [d + 1] and uj G S. Now let A G SoIa {{i,j),uj,p), then: 


inf sup Q") $,j-,(A,y,Z) 



d+1 

~ ^ ^ AmWm . 
m—1 


Let {i,j) G Sw, I G [d+1] and oj G S. From Corollaryfor all A G SoIa {{i,j),l,oj), we can rewrite for all 
V G ag follows: 

d+1 d+1 

(^) = E - E (24) 

m—1 m—1 

We remark that i(w) = F^i{uj). 

From the first statement of Prop, [^and the second assertion of Prop. the most precise overapproxi¬ 
mation of TZ (with these quadratic functions) is given by: 


w = infjw G I F'^iuj) < w} 


From Tarski’s theorem, oJ is the (finite) smallest fixed point of F^. So we are looking for the smallest fixed 
point of F^. The smallest seems difficult to obtain and since any vector uj such that F'’^{uj) < uj furnishes 
a valid but less precise overapproximation of TZ, we perform a policy iteration until a fixed point is reached. 


5.2 Policy definition 

A policy iteration algorithm can be used to solve a fixed point equation for a monotone function written as an 
infimum of a family of simpler monotone functions, obtained by selecting policies, see |CGG~*~05l IGGTZ07] 
for more background. The idea is to solve a sequence of fixed point problems involving the simple functions. 
In the present setting, we look for a representation of the relaxed function: 

V(z,j) G S^, VZ G [d+ 1], = inf F;-, (25) 

where the infimum is taken over a set 11 whose elements tt are called policies, and where each function F’’’ 
is required to be monotone. The correctness of the algorithm relies on a selection property, meaning in the 
present setting that for each argument {{i, j),l,u) there must exist a policy tt such that Fj^i (w)=F-,(a;). 
The idea of the algorithm is to start from a policy 7r°, compute the smallest fixed point u of F'"°, evaluate 
F^ at point u, and, if w F'^{u), determine the new policy using the selection property at point u. 

Let us now identify the policies. Lemma shows that for all I G [d + 1], FJ^ can be written as the 
infimum of the family of affine functions FA, the infimum being taken over the set of A G When 

w G 5 is given, choosing a policy tt consists in selecting, for each {i,j) G Sw and for alH G [d + 1], a vector 
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A G SoIa We denote by 7 rij^i{uj) the value of A chosen by the policy tt. Then, the map in 


Equation ( [^ is obtained by replacing F^i by F^j i appearing in Eq. (241. 
Finally, we define, for all I G [d + 1]: 

= sup{ sup_F^y (u), Xf} 

(z,j)GSw 

andF- = (E-,...,F,V). 

Now, we can define concretely the policy iteration algorithm at Algorithm 
Algorithm 1 Policy Iteration with PQL functions 
1 Choose 7r° G n. A: = 0. 


2 Define by choosing A according to policy using Eq. (241. 

3 Compute the smallest fixed point in of F'^'’. 

4 If G 5 continue otherwise return oj^. 

5 Evaluate if F'^iuj’^) = w* return otherwise take s.t. F'^{uj^) = (w^). Increment 

k and go to 2. 


5.3 Some details about Policy Iteration algorithm 


Initialization Policy iteration algorithm needs an initial policy. Recall that we have assumed that 
computed from an optimal solution {'P,W,U,Z,a,f3) of Problem (PSD I such that a > 0. The first 
is given by a choice of an element in SoR ((i,j),/,w°) where is defined by: 


L was 
policy 


Vfc G [d], col = /?> '“'d+i = a 


(26) 


with a and /? are extracted from (P, Z, a,/3). 

Proposition 8 The vector co^ satisfies F^(a;°) < a;°. 


Proof 6 In appendix. 


Smallest fixed point computation associated to a policy For the third step of Algorithm using 

Lemma F'^ is monotone and affine, we compute the smallest fixed point of F'^ by solving the following 

Linear Program see |GGTZ07l Section 4]: 

min iCfe s.t. < i(;| (27) 

Convergence In |Adj I4| , it is proved that policy iterations in the quadratic setting converges towards a 

fixed point of our relaxed functional. Here we establish a similar result (Th. [^. Combined with Prop. 
this fixed point provides a safe overapproximation of the reachable values set. 

Let consider the sequence (w*)i>o computed by Algorithm]^ If for some ^ G N, w* ^ 5 and G S, 
then we set = v/ for all fc > L 

Theorem 2 The following assertions hold: 

1. For all I G N, F^{w^) < re*; 
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2. The sequence {w’')i>o is decreasing. Moreover for all I G N such that ^ G S either ^ and 

F^{w^) = or ; 

3. For all I G N, for all k G [d + 1], < w^; 

4-. The limit of {w^)i>o satisfies: F^{w°°) < . Moreover if W k G N, G S then F^{w°°) = . 

Proof 7 (1) From Prop. F^{w^) < . Now, let I > 0 and assume G S, there exists tt* such that, 

F^ (w^) = and since F^ = we get F'^{w^) < F'^ (w^) = wf If ^ S, then there exists 

k gN, k < I — 1 such that G S and = w^, and thus by the latter argument we have F'^{w^) < w^. 

(2) Let I G N, if w’'~^ ^ S, . Now suppose G S. There exists S 11 such that 

F^{w’'~^) = F'^ < w''~^ and since is the smallest element of {v G | F'^ (v) < i;} then 

Now if , F^{w’'~^) = F^{w^) = F'^\w^~^) = F'"\w^) = = w''~^. 

(3) From the second assertion, for all I G N, w’’ < ui°. Moreover, for all k G [d + 1], < 

F^W) < w[. 

(4) First, w°° exists since (?i’i)/gN is decreasing and bounded from below (third assertion). Then, for all 

I G N, w°° < and thus since F^ is monotone (Prop. F^{w°°) < F^{w^) < wK Taking the infimum 
over I, we get F^{w°°) < w°°. Now we prove that w°° < F^{w°°). Let I G N. By assumption, G S 
and then, there exists G II such that F'^ ^ (ui*) = F^{w^). Moreover, and since F'^ * is 

monotone (^Prop. [^; = F'^ * ^ (w^) = F'^(w^). Now by taking the infimum on I, we mt 

w°° = inf; = inf; < inf; F'^{w^). Finally since F^ is upper semicontinuous (third point of Prop. |?j), 
then inffcP^('u;^) = limsup^. P^(w^) < P^(limfeW^') = P^(w°°). We conclude that w°° < P^(w°°). 


6 Example 

6.1 Example from [MFTMOO] slighty modified 

Consider the followinf PWA: X^ = [—1,1] x [—1,1], and, for all k G N: 


^fc+1 — 



if Xk,i > 0 and Xk ,2 > 0 
if XkA > 0 and Xk ,2 < 0 
if Xk,i < 0 and Xk ,2 < 0 
if Xk.i < 0 and 2 > 0 


with 


= 


A3 = 


-0.04 

-0.139 

-0.857 

0.491 


-0.461 

0.341 

0.815 

0.62 


Then, we have X^ = IR+ x K+, X"^ = K+ x 


A2 = 

= 

1, a 3 = Ml X: 


0.936 

0.788 

- 0.022 

0.758 


0.323 
-0.049 
0.644 
0.271 

and X^ = M 


From Prop. 


X It 

1011 

1001 

0110 

1100 


In = {1,2,3,4} and Sw = {(*, j) | S{i,j) = 1} with S = 

By solving Problem PSD[ we get a (optimal) PQL function L characterized by the following matrices: 

"1.5907 0.5907\ 


P3=( 

" 1.1178 

-0.1178\ 

, P2 = 

^-0.1178 

1.1178 ) 

P3 = 

f 1.3309 

-0.3309\ 

, P4 = 

1 -0.3309 

1.3309 ) 


Since a = /? = 2, then 72. C {a; € M^ | L{x) < 2}_C_{a; G 
and {a; G M^ I L{x) < 2} are depicted at Figure 


0.5907 1.5907^’ 
1.2558 0.2558\ 
0.2558 1.2558^ 

2 


la 


a ;||2 < 2}. The sets 72 (discretized version) 


Then we enter into policy iteration algorithm. From 


Equation (261, we define by: 


w( = 2.0000, = 2.0000, = 2.0000 
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_2 1 _ 1 _^_ I _ i 

- 2-10 1 2 
Xi 

(a) First overapproxima¬ 
tion found by | |PSD| l 



Xi 


(b) Final overapproxima¬ 
tion found by policy itera¬ 
tions 


Figure 1: (Discretized) 7Z in yellow and initial and last overapproximations of 7Z. 


Then we compute the image of by the relaxed semantics F^(w^) using semidefinite programming (see 
Eq. @). We check that is not a fixed point of and then the initial policy 7 r^((i,j), I, u>°) is the vector 
A extracted from the optimal solutions (A, Y, Z) of the semidefinite programs involved in the computation 
of For example, for (1,3) S Sw and I = 1, 7 r°((l,3), 1,= (0.0000,0.0000,0.0430)''', where the 

first two zeros are the Lagrange multipliers associated to Mi and M 2 and 0.0430 is the Lagrange multiplier 
associated to M(L^). We compute the smallest fixed point associated to 7r° using the linear program (27l: 




= 1.1036, wl = 1.2443, = 2.0000 


Moreover, at each step k, policy iterations provides auxiliary values which represent the overapproximations 
of the polyhedra 72. n V* n A* by ellipsoids of the form {x S | xf < 1 , X 2 < L{xi,X 2 ) < 

wfj 3 }. For example, for k = 0: 


wii.i = 0.0000, 

= 0.0573, 
wu.i = 0.3012, 


wii^2 = 0 . 0000 , 

u;i 3’2 = 0.0213, 
■^14 ,2 = 0.1447, 


mil 3 = 0.0000 

u;i 3’3 = 0.0213 
uii 4^3 = 0.1447 


Note that we found that for {i,j) 
is reduced to the singleton ( 0 , 0 ). 
iterations that for all k gN, xf f. 


= (1,1), wL 1 = w}j 2 = ^ij ,3 = 0 which means that 72 n n ^(-^^) 
The invariant found is depicted at Figure lb Finally, we find after two 
< 1, a'ffc < 1.2443 and L{xi^k,X 2 ,k) < 2. 


6.2 A (piecewise) afRne example 

We now consider the following PWA: = [0,3] x [0, 2] and for all k gN: 


f A^Xk + b^ if T(xfe) < c 
( A^Xk+b^ AT(xk)>c 


with 


= 

A^ = ( 


/0.4197 

1^0.5029 

-0.0575 

-0.3334 


-0.2859\ 1 _ /2.0000\ 

0.1679 J ’ ° ~ VS.OOOoj ’ 

-0.4275\ 2 _ /-4.0000\ 

-0.2682b ° ~ \ 4.0000 ) 


T = (3.0000 8.0000) and c = -3.0000 
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By Prop, g Sw = 12 = {(i, i), (i, 2 ), (2,1), (2, 2)} and In = {2}. 
PQL function L characterized by: 


Using Problem (PSD), we compute the 


/ 2.9888 -1.7890\ 

1^-1.7890 8.0295 ) 

and 

_ /2.7192 2.0930\ 

”1^2.0930 e.iiioj’ 


_ /-14.7283\ 

■ 1^-94.13477 

/ 5.5737 \ 
1^-16.4198^ 


and the invariant found is {a; G K2 | L(x) < 58.1165} and an upper bound over the square Euclidian norm 
of the state variable is 286.4932. We run the policy iteration to get finally after 4 iterations the following 
bound vector: 

wi = 41.8956, W 2 = 31.4449, = 58.1165 

corresponding to the invariant set w{x S K2 | L{x) < rca}. 

We obtain interesting information during policy iterations running. At step fc = 0, when we select the 
initial policy, the SDP solver returns for all I = 1,2,3, i{w^) = —00 and from Prop. this implies that 

suPj^gT^nxin/i-pxi)^^ot feasible hence (1,1) 7 Sw. At iteration step fc = 1, the SDP solver 
provides for all I = 1,2, 3 , liw^) = —00 and from Prop. 6 ^this implies that sup^g 7 ^pjfipj 2 -i(j(- 2 ) p{A'^x+b'^) 
is not feasible hence (2,1) ^ Sw. Finally, Sw C {( 1 , 2), (2^)}. Recalling that 1 ^ In, we conclude that the 
system state variable only stays in and thus the system is actually equivalent to a constrained affine 
system. This information is computed automatically. 


7 Conclusion and Future Works 


We have developed a method to compute automatically by semi-definite programming precise bounds over 
the reachable values set of a piecewise affine system. The method combines piecewise quadratic Lyapunov 
functions to generate a first overapproximation and policy iterations used to reduce the initial overapproxi¬ 
mation. 

Future works could be to design a repartitioning method in order to improve the feasibility of Prob¬ 


lem (PSD I. Morevoer, we can think of apply the method to maximize a quadratic form over the reachable 


values set. 

Also, we conjecture that the presented policy iterations algorithm provides the most precise overapproxi¬ 
mation considering bounding the square of coordinates variables. To reduce these bounds we have to choose 
a different set of quadratic functions. 
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Appendix 

In the appendix, we give details about the proofs of the propositions. 


Proposition 9 Assume that Problem (PSD) has a feasible solution {V,W,U, Z,a, P). Then: 

1. The family V defines a PQL; 

2. There exists {V,yV,U, Z,a, P) satisfiying and ( |18[ ) if and only if Problem ( |PSD[ ) is feasible; 

3. For all {i,j) € Sw, 


F*'^M(Id,0,0)F* 


^ M(P%29%-a)+M(0,0,/?) 

On, On, ,nj 

Wf + Wi 


-E^P 


+ m^+U!f]EP ; 


4- We have sup ||a ;||2 < P; 
xex° 

5. //(P, W, W, Z, a, /?) is optimal and a > 0 then sup L{x) = a. 

xexo 


Proof 8 (1) The first statement follows readily from Corollary 0. 

(2) The "if" part is obvious. Let us focus on the "only if" part and let {V,yV,U, Z ,a, P) satisfiy¬ 
ing ([T§, (0. From Th. [7] /3 > 0. If ct > 0, the proof is finished. Hence, we suppose that a < 0 and 
let us prove that {V,W,U,Z,0, P — a) is feasible for Problem (PSD). First P — a > 0 since P > 0 and 
a < 0. Second, M{P\ 2q\ 0) - M(Id, 0, -(/3 - a)) - E^'^ {W; + W\JE^ = M{P\ 2q\ -a) - M(Id, 0, -P) - 
E^ (W^ + Wf) E^ P 0 by the fact that {V,W,U,Z,a,P) satisfies ( [T^ and thus (V,W,U,Z,0, P — a) sat¬ 
isfies ( p^ . Since a and P do not appear in ( |T7| ) {V,W,U, Z,0, P — a) satisfies 0- Finally, 


-M{P\ 2q\ 0) - E^^'^ + Z°f) 

= -M(P*, 2q\ a-a)- E^^'^ E^° 

= M(0, 0, -a) - M{P\ 2q\ -a) - £1*°^ (Z°^ + E^° 


We conclude that -M(P% 2q\ 0) - E^^'^ + Z^) E^° > 0 and thus {V, yV,U, Z, 0, P 

(3) Let (i,j) G Sw. Since j G T, 


a) satisfies (18). 


M{PP2qP-a) - M(Id, 0, -p) - E^^ {w^ + Wl^ E^ P 0 

and thus 

eP {M{Pp2qP-a)-M{ld,0,-P) 

-El'' {W^ + Ei^ P* P 0 

and 

FPM{Pi,2qi, -a)F* - F^E^'' (wf + EiE^ 

h P*'^M(Id,0,-/3)F* 

Hence: 

F*'^M(Id, 0 ,-^)P* 

^ -F^Ei'' (wi + Wl^ EiF^ + M{P\ 2q\ 0) 

-E^i'' (ufi + EP 
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Note that 0, —I3)F'^ = M(0, 0, —jS) and thus: 

F*'^M(Id,0,0)F* 

^ _y j 0p^ 2q\ 0) 

j + M(0, 0, 13) 


We conclude by the definition of E^^. 

(4) Since {V, W,U, Z, a, /3) defines a PQL function, then the result of Th. holds that is E C {x G | 
|| 2;||2 < P} o-nd since C E, sup 3 ,gj 5 fo ||a ;||2 ^ P- 

(5) Now assume that {V, yV,U, Z, a, P) is an optimal solution such that a > 0 and suppose that sup^j^js^o L(x) 
a. We remark that sup^^js^o i(a;) = sup^gjjj sup^gj^ipjj-o iv*(a:) and from Constraint (18), for all i G In, 
X® n C {a; I L'^{x) < a}. Hence for all i G In, sup 2 ,gj(-inj(-o H(x) < a and thus sup^jg^^o L(x) < a. Let 

e > 0 such that 7 = a — e > 0 and sup„,gj(-o L{x) < 7. Let us denote by N the matrix defined by Ni^i = 1 
and Nym = 0 for all {l,m) G { 1 ,..., d + 1}2\{(1, 1 )}. We have -M(P\ 2q\-j) - E^°'^ + Zf)’E^° = 

-M{L^)+-fN- E^°'^ (Z°^ + Z°^) E^° = (a- e)N - M(L*) - E^°'^ (Z°^ + Z°*) E^°. Let us remark since Ef^^ 
is equal to 1, that E^^'^NE^^ = TV. Thus, 

-m{P\ 2q\ - 7 ) - E^°'^ (Z°^ + E^° 

= -M(L^) + aN- E^°'^ (Z°‘ + eN + E^° . 


In a second time, 

M{P\ 2q\ - 7 ) - M(Id, 0, -P) - E^^ {Wf + Wf) E^ 
= M{P\ 2q\ -a) - M(Id, 0, -P) - E^'^ {Wf + W^) E'- 
+eN . 


From Constraint ( [T^ , M(P*, 2g% — 7 ) — M(Id, 0, —/?) — (Wf + Wf)E'' is positive semidefinite. We 
conclude that (fP,W,U, Z'P) with Z' = {(Zp^ + eN,Z^^ G x G In} is feasible and 7 + /3 = 

a + P — e thus {'P,W,U, Z,a, P) cannot be optimal. 


Proposition 10 The following statements hold: 

1. F{C{uj)) C C{uj) ^ F«(w) < oj; 

2 . nc inf{(7(0;) | w S R‘^+^ s.t. F#(a;) < wj; 

3. For all I G [d + I], F^^ fiuj) is the optimal value of quadratic program; 

4- For all k G [d], = max{( inf xifipfi sup x^)^} and if L is constructed from an optimal solution 

xexo 2,g;if0 


{V,yV,U, Z,a, P) of (PSD) such that a >0, then X^^^^ = a. 


Proof 9 (1) F{C{uj)) C C{uj) iff for all k G [d], supyg^(c(^)) yl < ujk and supj^g^(c(^)) L{y) < Ud+i- Now 
for all k G [d]; 

SUPy6F(C(..)) vl 

= sup{supygA(C(<^)) yl^ SUPj^g^o yl) 

= sup{sup(^pgg^ sup yl, sup yl} 

y—A^x+h^ , 


yGXO 


= 


and 


SUPy6F(C(a;)) Hy) 

SUp{suPygA(C(c;)) SUPj^g^o L{y)} 
sup{sup(,pgg^ sup V{y), sup L{y)} 


Fhi^) 


sup 

y—A^x-\-b ^, 
x^C{uj), x^X"^^ 


yexo 
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(2) From Eq. (19), 7 ^ C inf{C(w) | oj G F'^(C{uj)) C C'(w)}. We conclude using the first point. 

(3) Obvious. 

(4) Let k G [d]. Since is compact and x i-G Xk is continuous then there exist z G and u G X° 

such that Zk = Uk = snp^^x^ ^k- Hence zj^ < X}^ < Uk for all x € and thus for all 

X G X°, x\ < max(z^,M|). Since z and u belong to X^, then = max(z^,u^). We have assumed that 
{V yW ,U, Z,a, P) is an optimal solution of Problem (PSD) and a > 0 then X^\l) = a from Prop. 

Proposition 11 (Safe overapproximation) The following assertions are true: 

1. For all I G [d + 1], F/^ is the optimal value of a SDP program; 

2 . pit < prK ^ 

Proof 10 (1) Obvious. 

(2) We have to prove that for all k G [d + 1], for all oj e f,{Lu) < F^f.{uj). We do the proof for 

the case k = d + 1. The other cases follows the same proof constructions. 

Applying the weak duality theorem, we obtain: 


^ijyd+ii^) < in| sup L^(/*(x)) + ^ Afe(wfc - xl) 


AgR:(+" x&Xio 


fc=l 


F 


< 


< 


< 


inf 
A,77 


S. t. 


inf 
A,17 


s. t. 


inf 

\,v,Y,z 


s. t. 


+Ad+i(wd+i — L’'{x)) 


Using Lemma^and Corollary^we get: 


Vx e xu, 

d 

77 - V{f\x)) - ^ Afc(a;fe - xl) 
-Xd+i{uJd+i - L\x)) -p{f\x)) > 0 
A > 0, ?7 S K 

M ^77 - L^{f{x)) - Xk{uJk - xl) 

\ k—1 

-Xd+i{iXd+i - L\x))) G Cd+i 

\ g TD><i+l 


> 


77 e 


1 


M [ 77 - L\f\x)) - Y ^kiujk - xl) 




-Xd+iitVd+i - L\x))) - + Z)E^^ 

P 0 


A e R‘l+\ 77 e 


y > 0, ZYO 
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Now from Eq. (|^ and since A —)■ M(A) is linear, we have: 


M^- U{P{x)) - ^ Afc(wfe - xl) - Xd+i{^d+i - L\x))^ 
d-\-l d 

= {V-Y. + Y, ^kMk + Xd+iMl 

k^l k^l 

d+1 

= {v-Y Z) + + Z)E^^ 


fc = l 


Finally: M{q-L3{f{x))-J2k=i^kii^k-xl)-Xd+i{uJd+i-L''{x)))-E^F(Y + Z)E^^ = {r]-J2kt\ ^kUJk)N- 
^ij,d+iiX,Y, Z). SineeF^i is the infimum ofrj over the constraint [rj — XkUJk)N — ^ij^d+i{X,Y, Z) Y 0, 

A G 77 G IR, y > 0 and Z YO, this achieves the proof. 


Proposition 12 Let {i,j) G Sw, ^ G [d+ 1], A G The following statements are true: 

F F^ i is affine; 

Z FY Fh and F^ are monotone; 

3. FJji and FJ^ are upper semi-continuous. 


Proof 11 The first assertion is straightforward from Equation (22). The function w 1 —>■ FX ^ (^w) is monotone 
from the positivity of X and the two last functions comes are monotone as the supremum of monotone 
funetions. The funetion w i-G F^[{w) is upper semi-eontinuous as the infimum of eontinuous funetions and 
w I—>■ Fj^iw) is upper semi-continuous as the finite supremum of upper semi-continuous functions. 


Proposition 13 The vector uj^ satisfies F'’^{ixP) < . 

Proof 12 From Prop. we have for all k G [d], < /3 = and = a = 

Then it suffices to prove that for all I G [d + 1], for all {i,j) G Sw, FF^{pj^) < uP. We can show it by 
proving that for all Z G [d + 1], for all {i,j) G Sw, there exist A > 0, P > 0 and Z YQ such that: 


d+l 

(w? - Y Xku:k)N - z(A, Y,Z)Y0 

k=l 


Let us define X by A^+i = 1 and A^ = 0 for all k G [d]. Let {i,j) G Sw. 


Recall that {V,W,U, Z,a, /3) is an optimal solution of Problem (PSDi. Let I = d+l, and let us extract 
and C/^ from U, then we have: 


(wS+i - Efctl Xku:k)N - $.,.;(A, U;p 
= + Ml - + u':^)EP 


We conelude that {oj^.!^^ — 'Yl,k=i Xk^k)Ff—^ij^i(X, UfP U^) Y 0 since {V, yV,U, Z, a, ff) is an optimal solution 
of Problem ( PSD[ ) and thus satisfies ([^ 

SDP problem (21 1 and thus 


We conclude that X,UpPU^) is a feasible solution of the 


/ o„. o„ - „ - \ 

Let I G [d], Y = wiJ 


+ Uf’ and Z = 


^ g ’ ^ + U+ where and are extraeted 


from W and Uif and UV are extracted from lA. We have: 


- Ylt\Xk^k)N - $yv(A,y, z) 

= M(0, 0,13-a)- PMf- E^'^ {Y + Z)EP 
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Now, remark that M/ ^ M(Id, 0,0) and thus + M{P\ 2q\ -a) - {Y + Z)E^i + M(0, 0, /3) ^ 

—F®^M(Id, 0,0)F* + M(P®, 2g®, —a) — + M(0,0,/3). The right-hand-side sum of matrices 

is positive semi-definite from the second assertion o/Prop. ^ We conclude that {uji{p),X,Y,Z) is a feasible 

. .. .. „ . . .. . 


solution of the SDP problem (211 and thus F^fiuj^) < 
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